Head mounted display (HMD) devices utilize a display panel mounted in front of a user's eyes to display various types of content, including virtual reality (VR) content, augmented reality (AR) content, and the like. Eye tracking often is implemented in HMD systems to facilitate various functionalities, such as foveated imaging (also known as gaze-contingent imaging), eye-movement based user input or interaction, and the like. Conventional eye tracking mechanisms typically employ a complex arrangement of lenses and mirrors to capture an image of the eye, and from this image estimate a gaze direction of the eye. However, HMD device manufacturers strive to provide HMD devices with reduced form factors, and the complex optical mechanism required in conventional systems to provide this eye tracking function without occluding the display panel often inhibits implementation of a small form factor for the HMD device. Moreover, the algorithms implemented to estimate the eye gaze based on conventionally-captured eye images often are compute intensive and thus impracticable to implement in HMD devices absent a costly and power draining high-performance computing system.